Click any tag below to further narrow down your results
Links
@mmastrac on X
Matt Mastrac is wiring Google DeepMind's EmbeddingGemma2 into a fast, trainable /v1/systemone endpoint called Jev. He claims it runs in about 20ms on text and 100ms on images, and that a model this small can punch above its weight with the right harness.
- The endpoint answers in roughly 20ms for text and 100ms for images.
- The model can be trained in real time and can distill knowledge from a larger upstream teacher model.
- It is multi-modal by default, handling both text and images.
The author uses the rapid rise and fall of Jev, a small "system one" decision model from TypeSafe AI, to show how AI launches get inflated through name-dropping, big unverified claims, and jargon. He then argues that most recent progress has been incremental rather than transformative.
- Jev is essentially a general-purpose classifier that doesn't need much training data, but it can't see or reason, so calling it a "decision model" is a stretch.
- The "can't hallucinate" claim only means the output matches a schema, not that it's correct. Jev also turned out to be often wrong, and open-weight models match or beat it on cost and performance.
- The launch relied on a famous founder, a Jevons paradox reference, and an undocumented training method (RLCD), and a similar idea reportedly appeared about 1.5 years earlier.
Cloudflare is previewing Clairic, a model that pulls values out of unstructured text, computes them, or forecasts them in a single pass with no generated tokens. It's pitched as faster, cheaper, and less error-prone than using an LLM for the same structured-output jobs.
- Clairic returns typed values (spans, numbers, dates, durations, lists, distributions) rather than free text, so it can't produce a malformed or hallucinated answer; missing values come back as null.
- Every field is answered in one parallel pass with no output tokens billed, so asking for thirty fields costs about the same as asking for three, and responses usually land in under a second.
- Each answer carries confidence scores, character-offset spans back to the source, and 80% intervals for estimates, so you can see which outputs to double-check.
TypeSafe released Jev 1.14.0 with a split approach: stable improvements for production use and experimental "danger zone" features (image support, bounding box type, longer context) for developers willing to accept less reliability. The company is also sharing internal intelligence metrics while acknowledging their limitations.
- Image support is now composable across Jev's JSON interface (state, instructions, criteria), though multimodal performance is weaker than text-only and requires careful prompt decomposition.
- New "box" type returns bounding boxes without type errors, representing the first Jev type designed beyond binary decisions and signaling exploration into broader System One Model capabilities.
- Jev supports longer context lengths than previously enabled in production; developers can opt in via a danger zone flag, but TypeSafe intentionally limits context by default because reliability degrades at scale.
- Internal benchmarking data is being shared for the first time, limited to open-weight models only, with explicit caveats that the methodology will evolve and developers should run their own evals.
Sean Goedecke explains how to build real-time AI systems using "System One" models—fast classifiers that pick from multiple-choice options instead of generating arbitrary text. He shares two practical techniques: tiered goals (breaking decisions into hierarchical loops) and tournament sampling (narrowing large option sets through multiple passes).
- You can convert any LLM into a fast classifier by batching single-token outputs with structured choices, getting predictable inference timing without retraining the model
- Tiered goal hierarchies (strategic → tactical → specific targets) let System One models handle complex tasks like game-playing by providing short-term context, similar to how LLM reasoning chains work
- Tournament sampling outperforms absolute scoring for large option sets—the Wikiracing demo found the optimal 3-link Wikipedia path by sampling 100 links at a time rather than trying to rank all 1,000+ options at once
Fast, general-purpose classifiers like Jev can be used as temporary solutions for specific tasks, then replaced with cheaper, faster custom classifiers trained on the input-output data the Jev instance generated. This creates a practical path for engineering teams without ML expertise to eventually build optimized task-specific models.
- System One models solve the immediate problem of deploying classifiers without ML expertise or large datasets — you just prompt them like an LLM
- Once a System One model proves a feature works, you can collect its decisions as training data to build a specialized classifier that's faster and cheaper to run
- This pattern lets teams validate whether a feature is worth building before investing in custom ML development, reducing risk and upfront cost